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Quasi-Kernel Polynomials and Convergence Results 
for Quasi-Minimal Residual Iterations 


Roland W. Freund 


Abstract. Recently, Freund and Nachtigal [9l have proposed a novel polynomi- 
al-based iteration, the quasi-minimal residual algorithm (QMR), f “ r “‘ vln 8 8 e "' 
eral nonsingular non-Hermitian linear systems. Motivated by the QMR method, 
in [6 we have introduced the general concept of quasi-kernel polynomials, and 
we have shown that the QMR algorithm is based on a particular instance of 
quasi-kernel polynomials. In this paper, we continue our study of quasi-kernel 
polynomials. In particular, we derive bounds for the norms of quasi-kernel poly- 
nomials. These results are then applied to obtain convergence th^rems both 
for the QMR method and for a transpose- free variant of QMR, the 1 r QMK 
algorithm. 


§1. Introduction 

Many iterative algorithms for solving large nonsingular systems of linear equa- 
tions 

Ax = b (1) 

are based on polynomials. For example, the classical conjugate gradient algo- 
rithm [13] for Hermitian positive definite matrices A and its various extensions 
to non-Hermitian linear systems all fall into the category of polynomial-based 
methods. For a survey of iterative methods, we refer the reader to [7] and the 
references given there. 

Typically, polynomial-based algorithms with optimal convergence prop- 
erties correspond to polynomials that are characterized as solutions of con- 
strained least squares approximation problems of the form 


min (ip, ip). (2) 

0 ) = 1 

Here and in the sequel, we use the notation 

■p„ := {ip( A) = 70 + 71 ^ "b ■ 4 * ■+■ 7n-^ | 7oi7i> • • • •> 7n £ C} (3) 

for the set of all complex polynomials of degree at most n. Moreover, (•, •) in 
(2) is an inner product that is positive definite on V n . The exact solutions of 
problems of the form (2) are normalized kernel polynomials, and they can be 
expressed in terms of the orthogonal polynomials associated with the inner 
product {-,*). 

For the special case of Hermitian matrices A, only inner products on 
the real line arise, and then the corresponding kernel and orthogonal poly- 
nomials satisfy three-term recurrences. As a result, optimal polynomial-based 
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algorithms for Hermitian linear systems can be implemented using only three- 
term vector recursions. The situation is less satisfactory for the case of general 
non-Hermitian matrices A. Here, inner products in the complex plane arise, 
and the corresponding complex kernel and orthogonal polynomials no longer 
satisfy short recurrences. Consequently, optimal polynomial-based methods 
for non-Hermitian linear systems in general involve long vector recurrences. 
Usually, it is too expensive to run the full versions of these optimal algorithms. 
Instead, restarted or truncated variants are used, which often leads to rather 
slow convergence. 

Recently, Freund and Nachtigal [9] proposed the quasi-minimal residual 
algorithm (QMR) for solving general nonsingular non-Hermitian linear sys- 
tems (1). The QMR method produces approximate solutions of (1) that are 
characterized by a quasi-optimality condition, namely a quasi-minimization 
of the residual norm. The point is that the QMR algorithm — unlike optimal 
polynomial-based iteration schemes — can be implemented using only short 
vector recurrences. 

Motivated by the QMR method, in [6] we have introduced the general 
concept of quasi-kernel polynomials, and we have shown that the QMR algo- 
rithm is based on a particular instance of quasi-kernel polynomials. Moreover, 
in [6] some general theory for quasi-kernel polynomials was given, such as re- 
currence relations and a characterization of roots of quasi-kernel polynomials 
as generalized eigenvalues. 

In this paper, we continue our study of quasi-kernel polynomials. In par- 
ticular, we derive bounds for the norms of quasi-kernel polynomials. These 
results are then applied to obtain convergence theorems both for the orig- 
inal QMR method and for a transpose-free variant of QMR, the TFQMR 
algorithm [5]. 

The remainder of the paper is organized as follows. In §2 we briefly re- 
call the definition of quasi-kernel polynomials. In §3 we present bounds for 
the norms of quasi-kernel polynomials. We then turn to non-Hermitian linear 
systems (1), and in §4 we describe the general setting of polynomial-based 
methods for the iterative solution of such systems. Finally, in §5 and §6 we 
consider the QMR and TFQMR algorithms, respectively, and we give conver- 
gence results for both methods. 

Throughout the paper, all vectors and matrices are allowed to have real or 
complex entries. As usual, M T = [m*j ] and M H = [m*j] denote the trans- 
pose and the conjugate transpose, respectively, of the matrix M = [mjk ]. We 
use the notation <r max (Af) and <r m j„(A/) for the largest and smallest singu- 
lar value, respectively, of the matrix M. The vector norm ||x|| 2 = Vx H x is 
always the Euclidean norm, ||M ||2 = cr max (M) is the corresponding matrix 
norm, and for nonsingular square matrices M, k 2 (M) = <x max (M)/<7 m j n (M) 
is the Euclidean condition number. Moreover, for square matrices M, we de- 
note by A (M) the set of all eigenvalues of M. In addition to (3), we use the 
notation Voo for the set of all complex polynomials. The symbol 0 will be used 
for the number zero, the zero matrix, and the polynomial t/>(A) = 0; its actual 
meaning and, in the case of the zero matrix, its dimension will be apparent 


Quasi-kernel polynomials 

from the context. Finally, /„ denotes the n x n identity matrix. 

§2. Quasi-Kernel Polynomials 

We consider constrained approximation problems of the form (2), where {•, •) 
is assumed to be a given positive semidefinite inner product on Vooi i- e -> f° r 
all <fi, <f 2 , ift € 'Poo and 71, 72 € C: 

{ 71^1 + 72^2, V>) = 7i(^i» V’) + 72(^2 ? V’)i 

{<?,'!’) = 

(<^¥>) > 0. 

Notice that we do not require (y >,<£> ) > 0 for ip ^ 0, i.e., (•, ■) is not assumed 
to be positive definite. It will be convenient to rewrite (2) in the form 

min ||V>||- W 

*p£V n : <Ho)=i 


Here and in the sequel, 

IMI :=V,v>> 1/2 , 


always denotes the seminorm induced by (•,•). 

If {•, •) is positive definite on V n , then there always exists a unique optimal 
solution rp* of (4). It is well known (see, e.g., [21, Chapter XVI], [1, Chapter 
I], [20], or [6, Corollary 3.3]) that </>* is just the suitably normalized nth kernel 
polynomial associated with {•,•)• Furthermore, is given by 




e;=o ^>(oVi( A ) 
e; = 0 i^(o)i 2 ’ 


(5) 


where p } € Vj, j = 0, 1 , . . . , n, is any set of orthonormal polynomials with 
respect to {•,•), i.e., 

(‘ Pj,<Pk ) = 

In the following, we will refer to ip* as a true nth kernel polynomial. 

Roughly speaking, quasi-kernel polynomials are approximations to true 
kernel polynomials, which are obtained from a set of arbitrary basis polynomi- 
als, rather than a set of orthonormal polynomials. More precisely, the setting 

is as follows. Let II j := {v?j}/=o> where J can either be a finite inte S er or 
equal to 00, be a given set of polynomials 

(pj^Vj with deg <fj = j, (®) 


J 0, if j ± k, 

\l, if j = k. 


which span Vj. We note that, by (6), each polynomial 1 < j < J, has 

a unique representation of the form 


j 

hk+ij<Pk(X), where h i+ U ± °- 

k = 0 


(7) 
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In the sequel, we always assume that n E {1,2, . . . , J), if J is finite, respec- 
tively n€{l,2,...},ifJ = oo. Moreover, we denote by 


H„ 


h\\ h\2 hi„ 

/i2i '• ^ 

0 : 

• "• ^r»,n— 1 hr* n 

. 0 • • • 0 ^n+l,n 


( 8 ) 


the (n + 1) x n upper Hessenberg matrix, which contains the recurrence coef- 
ficients hk+i t j from (7) as entries. We remark that, in view of (7), all subdi- 
agonal elements hj + ij in (8) are different from zero, i.e., H„ is an unreduced 
upper Hessenberg matrix. 

It follows from (6) that any polynomial xj) n £ V n with t/> n (0) = 1 can be 
parametrized in the form 


n — 1 


V’n(A) = 1 - A ^2 Cj+iVjWi where z := 
i=o 


Ci 

Cn 


€ C n . 


In [6] we have shown that each polynomial (9) satisfies 

I'M = ((<i„ - H„z) H G„(d„ - H„z))' n 


<(l|Gn|W’ / 2 ||<i„-JM| 2 . 


Here 


is the Gram matrix of tpor <Pi > • • • > Vn* and 

d n := [ l/v?o 0 0] T G C n+1 . 


( 9 ) 


( 10 ) 

( 11 ) 

( 12 ) 


Note that, by (6), ipo is a nonzero constant. 

The estimate (10) is the basis for the definition of quasi-kernel polyno- 
mials. We choose the parameter vector z in (9) such that, instead of ||0„ || , 
only the second factor in the upper bound in (10) is minimal. This leads to 
the following definition. 

Definition 1. [6, Definition 3.1]. Let z n £ C n be the solution of the least 
squares problem 


II d n - tf„z „||2 = min ||<f„ - H n z\\ 2 . (13) 

ze C" 

The polynomial tp n given by (9) (with z := z n ) is called the nth quasi-kernel 
polynomial (corresponding to the inner product (•,•) and derived from n j). 

Recall that H n is an unreduced (n + 1) x n Hessenberg matrix, and thus 
it has full rank n. This guarantees that (13) always has a unique solution z n . 
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In [6] we have established some general theory for quasi-kernel polynomi- 
als, such as recurrence relations and a characterization of roots of quasi-kernel 
polynomials as generalized eigenvalues. For example, in [6, Theorem 3.2], it is 
shown that rp n satisfies the same relation (5) as true kernel polynomials, where 
the <p/s are now the given basis polynomials (6). In particular, if the polyno- 
mials in II j are orthonormal, then the quasi-kernel polynomial t/>„ coincides 
with the true kernel polynomial. 


§3. Bounds for the Norms of Quasi-Kernel Polynomials 

In this section, we present some bounds for the norm ||V’n|| of the nth quasi- 
kernel polynomial rp n . 

First, note that, from (10) and (13), one obtains the estimates 


. jwi - iwi 


< (||G n || 2 ) 1/2 min ||d„ - H n z \\ 2 . 


(14) 


For later use, we remark that the first factor of the upper bound in (14) 
satisfies the inequality 


(i|G„|| 2 ) 1/2 <Vn + 1 max ||^j||, 


(15) 


which follows from 


||G n || 2 = max y H G n y 
yeC n+1 :||j|| 2 =i 

and the Cauchy-Schwaxz inequality. 

Clearly, the Gram matrix G n defined in (11) is positive definite if the 
inner product {•, •) is positive definite on V n . In this case, the condition number 
k 2 (G n ) of G„ is a measure for how far ||V’n|| can be from the minimal value 
W II Of (4). 

Theorem 2. Assume that the inner product (-,-) is positive definite on V n , 
and let i/>% be the true nth kernel polynomial. Then: 

11^511 < 110.11 <(*«(G.)) ,/J |l^ II- (16) 

In particular, if G„ = / n - 

Proof: The polynomial is the optimal solution of the approximation prob- 
lem on the left-hand side of (14). This implies the first inequality in (16). 

Let be represented in the form (9), and let z„ 6 C n be the corre- 
sponding parameter vector. Using the first relation in (10), it follows that 

110*11 = (( d » - - ff »^))‘ /2 

> K,m(G„))' /2 |K-ff„4ll2- 


(H) 
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On the other hand, from (14), we have 

HtMl < (^max(G„)) 1/2 min ||d„ - H n z\\ 2 

*eC (18) 

<(<r m UGn)) 1/2 \\dn-H n z«\\ 2 . 

Finally, by combining the estimates (17) and (18), we obtain the second in- 
equality in (16). I 


Next, we show that the upper bound in (14) can be related to certain 
constrained approximation problems. This result will be used in §5 and §6 to 
derive convergence theorems for QMR and TFQMR, respectively. 

We use the following setting. Let m < J be an arbitrary, but fixed integer. 
We denote by Ai, A 2 , . . . , A/ e C the distinct zeros of the mth basis polynomial 
cp m , and fij is the multiplicity of Xj. Note that 
l 

<Pm( A) = 7 JJ(A - A j)*’ for some 7 € C, 7^0. 
j= 1 

Furthermore, we denote by 

H:=[I m 0 ]H m (19) 

the m x m matrix obtained by deleting the last row of the mth Hessenberg 
matrix H m in (8). Using (7) and (19), one easily verifies the identity 

A$(A) = $(A)tf + [0 ••• 0 fc m+ i,mP*(A)], (20) 

where $(A) is a row vector defined by 

$(A) := [v? 0 (A) ••• v? m _i(A)]. 

By setting A = A j in (20), we deduce that the zeros of <p m axe just the 
eigenvalues of H, i.e., 

{A 1 ,A 2 ,...,A ; } = A (H), (21) 

and each A j is an eigenvalue of algebraic multiplicity fij. Next, we recall that 
H m (and thus H) is an unreduced upper Hessenberg matrix. This implies that, 
for each Xj, the matrix H — Xjl m has rank m — 1, and hence the eigenspace 
corresponding to A j is of dimension 1. As a result, there is exactly one Jordan 
block for each eigenvalue of H. Therefore, H has the Jordan canonical form 

J = SHS~\ (22) 


where S is a nonsingular m x m matrix and 

J = diag(J(A 1 ),J(A 2 ),...,J(A / )) 
is a block diagonal matrix with Jordan blocks 

0 10 ••• 01 


(23) 


. 0 


J(M) = + Wj. Nj ■■= 

I . 

0 1 

.0 0 

After these preliminaries, our result can now be stated as follows 


0 


6 IR^ . (24) 
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Theorem 3. Let m < J be a fixed integer, and let H be the m x m matrix 
defined by (19) with spectrum (21). Let J = 5H5" 1 be the Jordan normal 
form (22), (23), with Jordan blocks (24). Then, for n = l,2,...,m - 1, the 
nth quasi-kernel polynomial tp„ satisfies 


IV’ol 


where 


(l|Gn||2) l/2 £ n, 

(25) 

max ||V’(d(A J ))|| 2 . 

(26) 

= 1 ,...,/ 


max |V>(A)|. 

(27) 


Moreover, if H is diagonalizable, then: 


Proof: Let n G {1,2, . . . ,m - 1). By using (12) and by setting u : = <p 0 z, we 
obtain from (14) the estimate 


llV-nll < A (l|G „|| 2 ) ,/2 

I^ol 


where 


K := min ||e<" +1) - H n u\\„ e<" +,, :=[l 0 0] T € R" +1 . 

In view of (19), H n is the (n + 1) x n leading submatrix of H , i.e., 

H = 

and we have the relation 


(28) 


(29) 


H n * 
0 * 


H 


u 


H n u 

0 


0 


for all u € C r 


(30) 


Next, using the fact that H is an unreduced upper Hessenberg matrix, we 
deduce that 


| “ ti € C”| = Mff)e ( r' | * 6 r. 




(31) 


By means of (30) and (31), the quantity in (29) can be rewritten in the 
form 


i? „= min ||ej m ^ — <j>(H)e\ m ^ || 2 = l|V , (-^) e i ^ II 2 - (^2) 

<!>£Vn-\ V-67 > n:V'(0) = l 


_(»») 1 


.("*>11- _ 

With (22)-(24) and since ||eS m) || 2 = 1, it follows that, for all \p 6 Poo, 


U(H)e[ m) \\ 2 < k 2 (S) U(J)h = k 2 (S) max U(J(\j))\\ 2 . (33) 
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By combining (28), (32), and (33), we obtain the desired estimate (25), (26). 

Finally, if H is diagonalizable, then all Jordan blocks J(A ; ) = [Aj] are 
of size 1, and (26) reduces to (27). ■ 

We remark that the norms in (26) can be estimated by means of the 
inequality 

I*j(*i))lb < E i! * < 0 ( A i) • ( 34 ) 

i= 0 

which is an immediate consequence of the standard relation 

N-i 1 

V’(AAy)) = £ 

i=0 

Here Nj is the nilpotent matrix defined in (24), and denotes the *th 
derivative of ip. 


§4. Polynomial- Based Matrix Iterations 

We now return to systems of linear equations (1). From now on, it is always 
assumed that A in (1) is a nonsingular non-Hermitian N x N matrix. 

Many iterative algorithms for solving linear systems (1) are polynomial- 
based methods: they produce approximations x„ to A~ l h of the form 

x n = *o + <f>n(A)r 0 with <t> n€P n -i, n = l,2, .... (35) 

Here xq 6 is any initial guess for the solution of (1), and tq := b — Ax o. 
We remark that the residual vector r n := b — Ax n corresponding to the nth 
iterate (35) of any polynomial- based algorithm is given by 

r„ = tp n (A)r 0 , where tp n ( A) = 1 - A<£ n (A). (36) 

Note that 

ipn G V n and ^>„(0) = 1. (37) 

Generally, any tp n that satisfies (37) is called an nth residual polynomial. 

The goal in designing a polynomial-based method is to choose at each 
step the residual polynomial rp n in (36) such that some norm of r„ is as small 
as possible. A standard approach [18] is to require that the Euclidean norm 
of r n is minimal, i.e., 


|r " l|2= V.^”‘*'(0)=. ll, '’ < ' 4)r " 112 ' 


(38) 


By setting 


:=r(f(4>(A)) H tp(A)r 0 and IWl := {«/■’, V’)' /2 , 


(39) 
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we see that (38) is an instance of a constrained polynomial approximation 
problem of the form (4). We remark that other strategies (see [17,19]) for 
choosing the residual polynomial t/’n in (36) also lead to problems of the form 
(4), e.g., with inner products given by 


:= jj </?(A)t/>(A)u;(A) da or (v^V 7 ) := ¥ J (A)t/’(A)w(A) |dA| . 

Typically, G C C is a compact set containing the spectrum A(A) of A or some 
approximation to A(A), and C is a curve bounding such a set G. 

In the remainder of this paper, we always assume that (•, •) is the inner 
product defined by (39). Note that the corresponding polynomial norm ||-|| is 
equivalent to the Euclidean vector norm in C N , in the sense that 

\\rp\\ = || v || 2 for all v — *fi(A)r o with %{> € 'Poo • (40) 


Finally, we call a polynomial-based method optimsd if its residual vectors sat- 
isfy (38). We recall (see, e.g., [6]) that an optimal iteration scheme terminates 
after a finite number J 0 p t(< N) of steps with the exact solution xj opt = A b 
of (1). Furthermore, the residual vectors r° pt of an optimal polynomial-based 
method axe unique, and they axe given by 


r° pt = rp*(A)r 0 , n = 1,2, . . . , J op t, 


(41) 


where is the nth true kernel polynomial associated with (•, •). 


§5. Convergence Results for the QMR Algorithm 

Faber and Manteuffel [2] have shown that— except for a very special class 
of non-Hermitian matrices — optimal polynomial-based methods for general 
non-Hermitian linear systems can be implemented only with long recurrences, 
which involve vectors from all previous iterations. Their result motivated the 
development of a quasi-optimal polynomial-based iteration scheme, the QMR, 
algorithm [9], which can be implemented using only short vector recurrences. 
In this section, we apply the results from §3 to obtain two convergence theo- 
rems for the QMR algorithm. 

The QMR iteration uses the Lanczos process [14] with look-ahead [16,8] 
to generate basis vectors of the form 

Vj = <f>j(A)r 0 , j = 0,1,..., Jl- ( 42 ) 


Here 

<j>j £ Vj with deg <f>j = j, j = 0, 1, . . . , Jl, (43) 

axe the so-called Lanczos polynomials, and Jl is the termination index of 
the look-ahead Lanczos algorithm. We note that Jl ^ Jopt- F° r a detailed 
description of the Lanczos process, we refer the reader to [8] and the references 

given there. 
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In [6] we have shown that the QMR algorithm is based on quasi-kernel 
polynomials. More precisely, the QMR residual vectors axe given by 


QMR 

n 


ip n (A)r 0 , n 


12 7 

J. , . . . , U , 


(44) 


where J :=■ Jl, and i/>„ is the nth quasi-kernel polynomial corresponding to 
the inner product (39) and derived from the normalized Lanczos polynomials 



h 

IM’ 




j = 0, 1,..., Jl - 1, 
j = Jl . 


(45) 


For later use, we remark that the condition number of the Gram matrix G n 
associated with the basis polynomials (45) can be expressed in terms of the 
extreme singular values of the matrix of normalized Lanczos vectors 


V„:= 


vo 


L *>0 2 


V\ 

IK lla 


Vn ■ 

IKIh . ' 


More precisely, for n = 0, 1, . . . , Jl — 1, we have 


(« 2 (C„)) ,/2 = (46) 

^min v 

The relation (46) is an immediate consequence of the identity 

V H G„v = ||V„yf , y 6 C” +1 , 


which follows with (11), (39), (42), (45), and (40). 

Next, we apply Theorem 2, and we rewrite, by means of (40) and (46), 
the bounds in (16) in terms of the residual vectors (41) and (44). This gives 
the following result. 

Theorem 4. For n = 1, 2, . . . , Jl — 1, the QMR residual vectors satisfy 

ll-ril* < l|r? MR || 2 < llryik. (47) 

In particular , r^ MR = r° pt if the columns of V„ are orthonormal. 

We remark that a different proof for (47), without using quasi-kernel 
polynomials, was given by Nachtigal [15, Section 4.3]. 

Next, we derive upper bounds for ||r ^ MR ||2 by applying Theorem 3 (with 
m = Jl) for the special case of the QMR quasi-kernel polynomials. 

In the sequel, the same notation as in §3 is used. We set m := Jl, and let 


H:=[Ij L 0] 77 j L 


(48) 
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be the Hessenberg matrix of recurrence coefficients for the basis polynomials 
(45). As before, we denote by Aj , A 2 , . . . , A/ the distinct zeros of We remark 
that, by (45), the polynomial <p m = <f>j L is just the last Lanczos polynomial. 
It is well known (see [16,12]) that each zero of <f>j L is also an eigenvalue of A . 
Therefore, together with (21), we have 

{A 1 ,A a ,...,A I }-A(ff)CA(A). (49) 

Furthermore, we will need the relation 

kol = l/||ro|| 2 , ( 50 ) 

which follows from (45) (with j=0) and (40), and the inequality 

(||Gn|| 2 ) 1/2 <v^TT, n = 1,2, . . . ,m — 1, (51) 

which follows from (15) and (45). 

Finally, by applying Theorem 3, we obtain the following convergence 
result for the QMR algorithm. 

Theorem 5. Let H be the J L x J L matrix (48) corresponding to the nor- 
malized Lanczos polynomials (45), and let (49) be the spectrum of H. Let 
J = SHS -1 be the Jordan normal form (22), (23) of H, with Jordan blocks 
(24). Then, for n = 1, 2, . . . , J L ~ 1, the QMR residual vectors satisfy 

||r? MR || 2 < ||ro|| 2 K 2 (S)vAm£„, ( 52 ) 


where 

e n := min max ||V’(J(Aj))|| 2 . 
4>ev n :*Ko)=i j=i 

Moreover, if H is diagonalizable, then: 


e n < min 

ip€.V n -.iP(0)=l 


max |tf<A)|. 
AGA (A) 


(53) 


Proof: The inequality (52) is obtained by rewriting (25) in terms of the QMR 
residuals (44) instead of t/>„, and by using the relations (50) and (51). The 
estimate (53) follows from (27) and (49). ■ 


We remark that, for the special case of diagonalizable H, the bound (52), 

(53) was first given by Freund and Nachtigal [9, Theorem 6.1]. 

Of course, the bound in (53) is not very practical since it would require 
the knowledge of all eigenvalues of A. However, often one knows some compact 
set G C C such that 

A(4)CG and 0 £ G. 

In this case, we can replace (53) by the estimate 

e n < min max|V>(A)|. (54) 

o)=i agg 1 

The solutions of constrained polynomial approximation problems of the form 

(54) are known explicitly for some “simple” sets, such as disks [22], certain 
ellipses [3,4], and line segments [11]. 
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§6. A Convergence Result for the TFQMR Algorithm 

The QMR algorithm involves matrix-vector multiplications with the coeffi- 
cient matrix A, as well as its transpose A T . This is a disadvantage for certain 
applications, where matrix- vector products A • v, but not A T • w, can be com- 
puted efficiently without ever explicitly generating the matrix A. This was 
the motivation for developing a transpose-free variant of the standard QMR 
method, the TFQMR algorithm [5,10], which involves matrix-vector prod- 
ucts with A only. We remark that the QMR and TFQMR methods are not 
equivalent, and the two algorithms produce different sets of iterates. 

In this section, we present a convergence result for the TFQMR algo- 
rithm. It appears that this is the first convergence theorem for a transpose-free 
Lanczos-based iterative method. 

In [6] we have shown that — like the QMR algorithm — the TFQMR meth- 
od is also based on quasi-kernel polynomials. Its residual vectors are given by 

r TFQMR _ ^ n (A) ro , n = l,2,...,J, 

where J := 2 Jl and */>„ is the nth quasi-kernel polynomial corresponding 
to the inner product (39) and derived from basis polynomials ipo, • • • > <PJ 
that are now defined as squares or products of Lanczos polynomials (43). More 
precisely, the basis polynomials are defined as follows: 

if j = 2k, k = 0, 1, . . . , Jl — 1, 

if j = 2k- 1, k = 1,2,..., J L - 1, 

(55) 

if j = 2J l - 1 and ||<£j l _i<£jJ / 0, 
if j = 2Jl - 1 and = 0, 

if j = 2 J L . 

We remark that, as in the previous section, Jl denotes the termination index 
of the look-ahead Lanczos algorithm. 

Again, we apply Theorem 3, but this time with m := 2 Jl- Let 

H' :=[/j 1L 0] (56) 

where JJj aL now denotes the Hessenberg matrix (8) containing the recurrence 
coefficients of the basis polynomials (55). Here, the notation H' is used to 
distinguish the matrix (56) from the Hessenberg matrix (48) associated with 
the QMR algorithm. 

Recall from §3 that the eigenvalues of H' are just the distinct zeros of 
the polynomial <p m . Now, from (55) (with j = 2Jl), we have that 


Vi = \ 


uir 

^ Jl— 1 ^ Jl 
||^Jl-1<£JlI 
<£Jl-1^Jl> 


Quasi-kernel polynomials 

is the square of the last Lanczos polynomial. It follows that 
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A(tf') = {Ai,A 2 ,...,A,}, 

where, as in §5, the Aj’s denote the distinct zeros of Hence, in view of 
(49), the eigenvalues of the TFQMR and QMR Hessenberg matrices H and 
H coincide, and we have 

X(H') = X(H) C A (A). (58) 

However, their algebraic multiplicities are different. Indeed, by (57), each 
eigenvalue A j of H' has even algebraic multiplicity 2 pj, while A j is an eigen- 
value of H with algebraic multiplicity Pj{~> 1). In particular, the blocks in the 
Jordan normal form (22), (23) of H' are now of the form 


-o 

J(Xj) = Xih N + Nj, Nj := ; 

.0 


1 

0 


0 o- 

1 •• : 

0 6 ir2 W x2m; ( 59 ) 

•• 0 1 

0. 


Finally, we note that, for the basis polynomials (55), the relations (50) and 
(51) hold too. 

Therefore, we can proceed as in the previous section, and by applying 
Theorem 3, we obtain the following convergence result for the TFQMR algo- 
rithm. 

Theorem 6. Let H' be the 2J L x 2 J L matrix (56) corresponding to the basis 
polynomials (55). Then, the eigenvalues Aj , A 2 , - - • , Aj of H 1 are the zeros of 
the last Lanczos polynomial <f> j L , and each Xj is an eigenvalue of even algebraic 
multiplicity 2 p,j. 

Let J = SH'S' 1 be the Jordan normal form (22), (23), with Jordan 
blocks (59). Then, for n = 1,2,. . . ,2J L - 1, the TFQMR residual vectors 
satisfy 

||rI FQMR || 2 < ||ro|| 2 /c 2 (5)V^n£„, 

where 

e„ := min max ||V'(*^(Ai))|| 2 - (50) 

Moreover, if all zeros of the last Lanczos polynomial <f>j h are simple, then: 

e n < min max (|0(A)| + |V ,/ (A)|) . (61) 

“ v-er„:V>(o)=i AeA(A) V 

We remark that the estimate (61) follows from (60) by using (58) and 
inequality (34) (with p,j replaced by 2). 
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